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I.  INTRODUCTION 


The  importance  of  this  project  is  tied  to  the  notion  that  data  is  as  important  as 
ammunition.  In  the  same  way  that  faulty  ammunition  can  bring  a  ruinous  outcome  to  an 
otherwise  perfect  mission,  poor  data  quality  can  lead  to  disastrous  results.  Conversely,  accurate 
data  can  lay  the  foundation  for  smarter  decision-making  at  all  levels  of  an  organization.  Good 
data  is  especially  critical  in  the  business  of  logistics — a  primary  activity  within  the  Joint 
Munitions  Command  (JMC). 

The  author’s  efforts  in  researching  this  thesis  focused  on  data  quality  measurements 
obtained  from  the  JMC  Enterprise  Integration  Data  Team,  which  has  been  charged  with  creating 
and  publishing  accuracy  measurements  for  all  critical  data  objects  and  with  establishing 
consistent  business  rules  to  be  applied  and  communicated  to  all  sites  within  the  JMC.  These 
sites  include  Letterkenny  Munitions  Center,  Blue  Grass  Army  Depot,  McAllister  Army 
Ammunition  Plant,  Tooele  Army  Depot,  Crane  Army  Ammunition  Activity,  and  Pine  Bluff 
Arsenal.  Although  regularly  scheduled  audits  are  performed  by  the  cognizant  business  process 
owners  at  each  of  the  JMC  component  organizations  on  a  variety  of  production-related  business 
elements  (such  as  routes,  planographs,  bills  of  material,  etc.),  the  author  has  limited  the  scope  of 
this  study  to  audits  of  bills  of  material  (BOM)  data. 

A,  PURPOSE  OF  RESEARCH 

The  author’s  purpose  in  this  study  was  to  determine  the  existence  of  data  quality  patterns 
or  trends  in  component  organizations  of  the  JMC.  In  order  to  make  this  determination,  the  author 
studied  data  quality  records  of  the  various  JMC  components  and  qualified  the  data  defects  by 
category.  Based  on  the  data  qualifications,  the  author  was  able  to  make  strategic 
recommendations  for  reducing  data  defects  across  the  JMC  enterprise. 

B.  RESEARCH  QUESTIONS 

In  order  to  accomplish  the  goal  of  this  research,  three  major  questions  must  be  answered: 

•  Can  the  data  quality  data  be  qualified  in  any  meaningful  way? 

•  Are  there  data  quality  trends  or  patterns  that  exist  within  the  JMC? 


1 


Can  a  strategy  be  applied  that  will  reduce  data  defects  across  the  JMC  enterprise? 


C,  BENEFITS  OF  THE  RESEARCH 

Today’s  U.S.  armed  forces  rely  heavily  on  information  to  win  the  fight  in  a  battle  space 
that  encompasses  land,  air,  sea,  space,  and  now  cyberspace.  The  Pentagon  has  invested  tens  of 
billions  of  dollars  in  automated  information  systems  (AISs)  to  gather,  store,  process,  and 
disseminate  mission-critical  information.  The  LMP  is  the  U.S.  Army’s  latest  AIS,  and  its 
purpose  is  to  manage  the  key  logistical  needs  of  the  Department  of  Defense  (DoD).  The  JMC  is 
deeply  involved  in  that  logistics  mission  by  providing  conventional  ammunition  to  the  Army, 
Navy,  Air  Force,  and  Marines,  and  it  seeks  to  leverage  the  LMP  to  optimize  its  capabilities  by 
placing  the  lowest  possible  burden  on  taxpayers.  However,  the  LMP  is  only  as  useful  as  the  data 
it  possesses.  With  so  much  riding  on  the  viability  of  the  LMP,  achieving  and  maintaining  quality 
data  is  imperative.  The  author’s  primary  aim  in  this  research  is  to  increase  understanding  about 
how  the  JMC  can  improve  its  business  processes  to  properly  manage  data  quality  and  realize  the 
full  potential  of  what  the  LMP  can  do. 

D,  LIMITATIONS  OF  THE  RESEARCH 

The  JMC  collects  data  quality  measurements  on  several  business  elements  across  its 
enterprise.  Assuming  that  the  organizational  processes  are  the  same  for  each  business  element, 
performing  an  analysis  of  each  business  element  would  be  superfluous.  For  this  reason  the 
author  chose  to  examine  only  one  element:  the  BOM.  Additionally,  since  the  JMC  has  only  been 
fielded  with  the  LMP  since  October  2010,  the  amount  of  audit  data  is  limited  in  depth. 

E,  METHODOLOGY 

The  author  began  this  research  by  contacting  the  director  of  the  JMC  Enterprise 
Integration  Data  Team  and  requesting  access  to  the  team’s  SharePoint  server.  With  this  access, 
the  author  downloaded  the  data  quality  scorecards  from  previous  audits  of  the  major  commands 
within  the  JMC.  The  author  did  not  know  of  the  existence  of  this  data  at  the  beginning  of  this 
project,  and  it  was  helpful  in  furthering  this  research.  The  data  served  as  a  means  of  quantifying 
the  accuracy  of  BOM  records  and  provided  insight  into  the  immediate  causes  of  defective  data. 
The  author  then  categorized  these  causes  as  training,  policy,  or  process  related  in  order  to 
develop  strategies  for  reducing  the  data  defects. 
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F.  ORGANIZATION  OF  THE  REPORT 

This  joint  applied  project  is  composed  of  five  chapters.  Chapter  I  contains  the  purpose  of 
the  research,  the  research  questions,  the  benefits  of  the  research,  the  limitations  of  the  research, 
and  the  research  methodology.  Chapter  II  is  a  literature  review  in  which  the  author  touches  on 
training  issues  raised  by  a  GAO  report  and  then  discusses  documentation  and  research  published 
with  regards  to  innovative  approaches  to  training,  which  this  study  finds  is  a  factor  in  improving 
data  quality.  In  Chapter  II  the  author  also  presents  an  overview  of  the  various  missions  of  the 
JMC  component  organizations,  background  on  the  LMP,  an  overview  of  data  quality,  and  the 
methodology  of  conducting  an  audit.  In  Chapter  III  the  author  presents  the  scorecard  and  audit 
data.  In  Chapter  IV  the  author  provides  analysis  of  this  data.  In  Chapter  V  the  author 
summarizes  the  findings  and  provides  recommendations. 

G.  SUMMARY 

In  this  chapter  the  author  provided  an  overview  of  what  this  project  seeks  to  accomplish 
and  tried  to  convince  the  reader  of  how  important  data  quality  is  to  the  mission  of  the  JMC  and 
to  the  U.S.  armed  forces  at  large.  In  the  next  chapter  the  author  presents  a  literature  review  and 
the  background  information  necessary  to  highlight  the  importance  of  this  subject  matter. 
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II.  LITERATURE  REVIEW 


A,  INTRODUCTION 

The  GAO  is  an  independent  ageney  of  the  U.S.  government  that  performs  investigations 
on  other  ageneies  of  the  federal  government.  One  of  the  GAO’s  speeifie  purposes  is  to 
determine  how  effieiently  government  programs  are  exeeuted.  To  learn  more  about  the  status  of 
the  Army’s  implementation  of  the  LMP,  the  author  studied  GAO  reports  on  the  subjeet. 

One  GAO  report  the  author  studied  was  released  in  April  2010  and  eritieized  the  Army 
beeause  its  “training  strategy  did  not  effeetively  provide  LMP  users  the  skills  neeessary  to 
perform  their  new  tasks”  (p.  2).  The  report  further  elaimed,  “Users  at  the  depots  stated  that  the 
training  they  reeeived  did  not  provide  a  realistie  environment  that  showed  them  how  to  perform 
their  expeeted  duties  and  did  not  always  match  their  new  responsibilities”(p.  2). 

Aspects  of  the  LMP  influenced  by  humans,  such  as  training  on  how  to  input  data,  can 
have  a  profound  impact  on  data  quality  because  of  the  risk  of  human  error.  If  the  source  data  is 
incorrect  or  missing,  the  subsequent  data  created  by  the  LMP  will  be  flawed  and  of  a  low  quality. 
Based  on  this  reality,  the  author  examined  documentation  and  published  research  on  innovative 
training  strategies  in  order  to  establish  a  foundation  for  this  research. 

B,  BLENDED  LEARNING  TRAINING 

With  the  development  of  any  new  system,  whether  it’s  a  stealth  bomber  or  an  enterprise 
resource  planning  (ERP)  system,  any  human  interaction  with  the  system  requires  a  training 
strategy.  One  innovative  strategy  is  called  blended  learning.  Instead  of  solely  using  the 
traditional  brick-and-mortar  classroom,  blended  learning  fuses  traditional  learning  with  a 
combination  of  methods,  including  digital  and  web-based  instruction.  The  result  is  a  richness  of 
learning  that  exceeds  what  any  one  method  could  yield  on  its  own.  In  fact,  students  who  mix 
online  learning  with  traditional  coursework  in  a  blended  learning  approach  have  shown  increased 
learning  over  students  who  attend  traditional  brick-and-mortar  schools  (Means,  Toyama, 
Murphy,  Bakia,  Jones,  2010). 

As  described  by  Plifka  (2011),  blended  learning  has  four  main  areas  of  instruction;  face- 
to-face  (F2F),  print,  digital,  and  web.  It  allows  for  each  area  to  have  subordinate  methods  of 
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instruction  that  can  also  be  blended.  These  subordinate  methods  may  inelude  distanee  learning, 
distributed  learning,  or  traditional  learning,  just  to  name  a  few  (see  Appendix,  Figure  1;  Plifka, 
2011). 

Aeeording  to  Plifka  (2011,  p.  15),  “it  is  important  to  know  all  of  these  methods  of 
instruction  to  fully  grasp  the  possible  [blended  learning]  combinations  that  can  be  used  to  ereate 
the  most  effeetive  training  program.”  Plifka  eontinues,  “it  is  also  important  to  take  into 
eonsideration  the  objeetive  of  the  course  or  program,  faculty  expertise,  student  ability,  and  the 
infrastrueture  and  available  resourees  of  the  organization”  (2011,  p.  15). 

The  advantages  of  blended  learning  inelude  the  following: 

•  enhaneed  opportunities  for  teaeher-student  interaetion  and  inereased  student 
engagement  in  learning  through  a  plurality  of  mediums; 

•  greater  flexibility  and  aceess  for  students  through  the  ineorporation  of  distanee 
and  web-based  instruction;  and 

•  greater  opportunity  to  provide  students  basie  information  early  via  web-based 
instruetion  in  advanee  of  more  detailed  elassroom  training.  (Plifka,  2011) 

In  addition  to  these  advantages,  blended  learning  is  eompatible  with  the  Army’s  interest 
in  training  strategies  that  are  meant  to  “optimize,  synehronize,  and  support  training  in  sehools 
and  in  units,  and  to  promote  self-development  training  in  order  to  produee  forees  eapable  of 
responding  aeross  the  speetrum  of  operations”  (Department  of  the  Army,  2007,  page  2). 

The  LMP  training  that  Army  depot  employees  reeeived  was  given  within  the  structure  of 
a  traditional  elassroom,  and  the  allotted  elass  time  and  course  format  did  not  allow  students  to  be 
instructed  on  basie  eoneepts  about  the  LMP  and  how  it  applied  to  their  job.  Had  the  blended 
learning  approaeh  been  utilized,  many  fundamental  aspeets  of  the  LMP  eould  have  been  learned 
via  web-based  elasses  during  whieh  individual  users  eould  have  digested  elementary  material  at 
their  own  paee  as  a  preeursor  to  more  in-depth,  hands-on  training  in  either  a  elassroom  or  venue 
that  approximated  the  real-world  work  environment  of  the  user. 
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C.  THE  ARMY  LEARNING  CONCEPT  2015 

The  concepts  contained  in  the  Army  Learning  Concept  2015  (ALC  2015)  is  similar  to 
blended  learning  in  that  it  makes  use  of  multiple  training  methods,  but  its  focus  is  on  being 
learner-centric.  In  other  words,  ALC  2015  puts  the  emphasis  on  adapting  the  training 
methodology  to  the  needs  and  learning  strengths  of  the  individual  student.  This  strategic 
training  initiative  is  described  in  detail  in  United  States  Army  Training  and  Doctrine  Command 
(TRADOC)  Pamphlet  525-8-2  (2011),  and  is  a  product  of  the  TRADOC,  which  is  a  major 
command  of  the  U.S.  Army  responsible  for  overseeing  the  training  of  Army  forces  (TRADOC, 
n.d.;  see  Appendix,  Figure  2). 

ALC  2015  (TRADOC,  2011)  was  developed  in  response  to  the  training  challenges 
presented  by  a  population  of  personnel  composed  of  a  diverse  mix  of  ethnic  backgrounds, 
generations,  sociological  backgrounds,  and  levels  of  education.  The  ALC  2015  strategy 
acknowledges  that  American  society  is  producing  a  generation  of  citizenry — from  which  the 
Army  will  draw  the  soldiers  of  tomorrow — that  will  have  significant  knowledge  gaps  in  reading, 
writing,  mathematics,  and  other  important  areas  due  to  failures  of  the  U.S.  educational  system. 
Given  that  the  Army  must  be  capable  of  fielding  a  force  that  can  defeat  any  adversary,  it  is 
incumbent  upon  the  Army  to  develop  a  strategy  of  quickly  and  effectively  training  its  personnel 
in  a  manner  that  accommodates  the  capabilities  of  the  individual  learner. 

Table  1  contains  a  succinct  comparison  between  traditional  training  strategies  and  those 
of  the  ALC  2015  (TRADOC,  2011). 
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Table  1 .  Comparison  Between  Traditional  Training  Strategies  and  ALC  2015  Strategies 


Traditional  Training  Strategy 

ALC  2015  Strategy 

Instruetor-led  and  struetured  in  a 
predetermined  fashion  that  is  inflexible  for 
meeting  individual  learner  needs 

“Possesses  an  infrastrueture  that  is 
eomposed  of  subjeet-matter  experts  and 
faeilitators  from  the  eenters  of  exeellenee  , 
a  digitized  learning  media  produetion 
eapability,  knowledge  management 

struetures,  and  polieies  and  resoureing 
models  that  are  flexible  enough  to  adapt  to 
shifting  operational  and  learner  demands” 
(TRADOC,  2011) 

Based  on  individual  tasks,  eonditions,  and 
standards,  and  primarily  delivers  only 
eoneepts  and  knowledge 

Promotes  learning  “  through  outeome- 
oriented  instruetional  strategies  that  foster 
thinking,  nurture  initiative,  and  provide 
operationally  relevant  eontext”  (TRADOC, 
2011) 

Rigidly  formatted  programs  of  instruetion 
(POIs)  that  do  not  readily  allow  for  the 
refleetion  or  repetition  needed  to  proeess 
fundamental  information 

Provides  a  learner-eentrie  framework  that 
is  “eontinuously  aeeessible  and  provides 
learning  at  the  point  of  need  in  the  learner’s 
eareer” 

Learning  proeess  begins  before  initial 
military  training  and  eontinues  throughout 
a  learner’s  eareer  via  digitized  learning 
eontent  (TRADOC,  2011) 

Leeture-based  instruetion  that  is  oftentimes 
passive,  one-way  eommunieation  and  that 
does  not  integrate  the  aeeumulated 
knowledge  of  learners’  past  experienees 

Provides  students  with  ehallenging  eontent 
through  a  mixture  of  live  teaehing  and 
teehnology  in  variety  of  venues 

Ineentivizes  individuals  to  pursue  learning 
that  supports  position  assignments  and 
eareer  goals  (TRADOC,  2011) 

In  short,  there  are  two  major  goals  of  the  ALC  2015  (TRADOC,  2011).  One  is  to 
improve  training  through  “outeome-oriented  instruetional  strategies  that  foster  thinking,  nurture 
initiative,  and  provide  operationally  relevant  eontext”  (TRADOC,  2011,  p.21).  The  other  is  to 
extend  this  rieher  learning  experienee  throughout  the  eareers  of  personnel  by  making  it 
eonstantly  available  via  network  teehnologies  so  that  previously  learned  eontent  is  always  readily 
aeeessible. 

The  training  assoeiated  with  the  LMP  was  given  within  a  traditional  format.  Army  depot 
employees  would  have  reeeived  an  enhaneed  learning  experienee  if  the  training  had  been 
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administered  within  the  framework  of  the  ALC  2015  (TRADOC,  2011)  approach.  Students 
would  not  have  been  subjugated  to  the  one-size-fits-all  approach  of  traditional  teaching  methods. 
Instead,  the  learning  environment  would  have  been  adapted  to  a  format  that  would  be  most 
beneficial  to  the  specific  learning  needs  of  individual  personnel.  Additionally,  as  time  moves  on, 
personnel  would  continue  to  have  had  ready  access  to  previously  learned  material  on  an  as- 
needed  basis  throughout  the  balance  of  their  careers. 

D,  LOGISTICS  MODERNIZATION  PROGRAM  BACKGROUND 

In  order  to  properly  couch  the  prior  discussion  on  training  strategies  within  the  context  of 
the  LMP,  it  is  beneficial  to  discuss  how  and  why  the  LMP  was  developed.  Prior  to  the  LMP,  for 
about  30  years  the  Army  had  used  the  Commodity  Command  Standard  System  (CCSS)  and  the 
Standard  Depot  System  (SDS)  to  support  the  Army’s  procurement  of  supplies  and  equipment. 
These  systems  were  managed  by  the  Communications-Electronics  Command  (CECOM;  Hill, 
2007,  p.  47).  Though  useful,  these  systems  were  not  as  capable  as  state-of-the-art  logistical 
planning  systems  being  developed  and  implemented  in  private  industry. 

In  1997,  two  important  documents  were  created  that  would  set  the  DoD  on  a  path  toward 
the  eventual  creation  of  the  EMP.  The  first  document  was  the  Quadrennial  Defense  Review 
(Office  of  the  Secretary  of  Defense  [OSD],  1997a).  In  the  Quadrennial  Defense  Review  a 
military  asset  management  plan  was  described  that  emphasized  “focused  Eogistics”  (OSD, 
1997a,  p.  4).  This  plan  articulated  DoD  intentions  to  take  advantage  of  information  technology 
breakthroughs  in  the  civilian  sector  and  apply  those  technologies  within  the  DoD  to  radically 
improve  supply  chain  management.  The  second  document  was  the  1997  Defense  Reform 
Initiative  (DRI),  chartered  by  Secretary  of  Defense  William  Cohen.  Its  purpose  was  to  study 
changes  that  the  DoD  needed  to  make  to  its  business  processes  in  order  to  become  more 
efficient.  The  initiative  also  “authorized  the  services  and  DOD  support  agencies  to  begin  IT 
projects  to  acquire  systems  that  will  help  the  Department  perform  ‘just-in-time’  logistics”  (Hill, 
2007,  p.  47). 

That  same  year,  in  response  to  the  DRI,  a  CECOM  project  team  was  charged  with 
assessing  SDS  and  CCSS.  Hill  provides  the  following  list  of  conclusions  about  the  team’s 
assessment  of  the  weaknesses  of  SDS  and  CCSS: 
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Lack  of  flexibility:  Process  changes,  regulatory  changes,  and 
reorganizations  within  and  between  user  commands  require  expensive 
and  extensive  data  conversions  and  programming  changes. 

Slow,  unfocused  reports:  Reporting  and  summarization  capabilities  are 
geared  to  workers.  Managers  and  executives,  with  their  need  for  easily 
specified,  flexible,  tailored,  and  rapid  generation  of  reports  and 
summaries  are  usually  frustrated  with  output  capabilities. 

Difficult  to  use:  The  system  is  not  user  friendly.  The  system  relies  on 
extensive  use  of  codes  to  provide  compact  storage  (a  holdover  from  the 
time  when  computer  storage  was  inordinately  expensive).  Users  are 
required  to  learn  codes  and  have  extensive  system  knowledge.  The 
system  lacks  adequate  data  edits  and  validations,  as  well  as  support 
functions. 

Expensive  to  maintain:  The  system’s  size  and  complexities  make  it 
difficult  to  manage  and  change  code.  Large  portions  are  based  on 
relatively  old  third-generation  programming  languages  and  flat  data 
structures  that  are  inflexible  to  change  and  inefficient  to  operate. 

Unresponsive:  The  use  of  batch  processing  precludes  timely  updates  to 
data  architecture,  flexible  data  retrieval  capabilities,  and  informed 
decision-making. 

Outmoded  database:  The  use  of  outmoded  database  systems  and 
architecture  result  in  rampant  data  inconsistencies,  data  duplication,  and 
the  lack  of  data  standardization. 

Expensive  to  operate:  The  system  requires  extensive  manual 
intervention  because  of  outmoded  data  and  system  architectures. 

Lack  of  cost-sharing:  The  Army  is  the  only  “bill  payer,”  precluding  the 
ability  to  leverage  existing  industry  investments  in  modem  logistics 
processes  and  IT.  (Hill,  2007,  p.  47) 


The  CECOM  project  team  also  recommended  that  the  AMC  outsource  the  development 
of  a  replacement  system  based  on  specific  performance  requirements  that  would  address  the 
shortcomings  of  SDS  and  CCSS.  Because  the  development  was  inherently  an  outsourcing 
activity,  the  Office  of  the  Secretary  of  Defense  (OSD)  directed  the  project  team  to  follow  Office 
of  Management  and  Budget  (0MB)  Circular  A-76  guidelines.  After  a  period  of  deliberation 
between  the  offices  of  the  primary  stakeholders  (which  included  the  National  Federation  of 
Federal  Employees  [NFFE]  and  the  AMC  Commanding  General),  the  Secretary  of  the  Army 
finally  granted  the  AMC  the  authorization  to  move  forward  on  the  outsourcing  procurement.  On 
December  30,  1999,  the  Computer  Sciences  Corporation  (CSC)  was  awarded  a  contract  to 
develop  a  system  for  replacing  SDS  and  CCSS.  That  system  came  to  be  known  as  the  Eogistics 
Modernization  Program  (Hill,  2007,  p.  48). 

Upon  contract  award,  CSC  immediately  went  to  work  on  the  FMP,  and  by  November 
2002  end  user  training  and  testing  was  underway,  with  the  testing  designed  to  determine  if  the 
FMP  met  the  requirements  established  for  it  by  the  AMC  (Caterinicchia,  2002).  Having  passed 
the  initial  testing,  the  first  FMP  deployment  occurred  in  February  2003.  In  October  2010  the 
final  FMP  deployment  was  fielded  to  the  JMC.  However,  a  variety  of  factors  have  prevented  the 
JMC  from  fully  benefiting  from  the  FMP’s  promised  functionality,  not  the  least  of  which  has 
been  the  fight  to  achieve  and  maintain  data  quality. 

E,  DATA  QUALITY  DESCRIPTION 

In  order  to  understand  data  quality,  it  must  first  be  established  that  its  nature  is  both 
multidimensional  and  hierarchical.  These  characteristics  are  most  evident  when  one  considers 
that  first  and  foremost,  the  data  must  be  accessible.  Second,  the  data  must  be  interpretable. 
Third,  the  data  must  be  useful.  Finally,  the  data  must  be  believable.  These  basic  requirements 
form  the  primary  dimensions  of  data  quality  (Wang,  Reddy,  &  Kon,  1992). 

When  considering  the  accessible  dimension,  at  least  one  prerequisite  is  the  need  for  the 
data  to  be  available.  Fikewise,  in  order  to  have  useful  data,  it  must  first  be  relevant.  The 
relationships  among  these  dimensions  and  sub-dimensions  are  depicted  (see  Appendix,  Figure  3, 
Wang  et  ah,  1992). 
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The  multidimensional  and  hierarchieal  nature  of  data  quality  provides  a  conceptual 
framework  for  understanding  the  characteristics  that  define  data  quality.  In  this  research  project 
completeness  and  accuracy  are  the  relevant  dimensions  for  assessing  the  quality  of  data  obtained 
by  the  JMC  Enterprise  Integration  Data  Team.  Component  organizations  of  the  JMC  generate 
the  data  quality  scores  as  one  measure  of  how  well  they  are  executing  their  missions. 

F.  MISSIONS  OF  THE  JMC  COMPONENT  ORGANIZATIONS 

The  value  of  the  data  being  utilized  within  the  JMC  is  tied  to  the  missions  performed  by 
the  component  organizations.  Therefore,  any  discussion  about  the  quality  of  data  would  be 
incomplete  without  at  least  a  brief  overview  of  those  missions.  The  following  sections 
summarize  the  missions  of  JMC  component  organizations  whose  BOM  data  was  used  in  this 
research. 

1.  Blue  Grass  Army  Depot 

Located  in  Richmond,  Kentucky,  Blue  Grass  Army  Depot  (BGAD)  is  a  “Strategic 
Mobility  Power  Projection  ammunition  depot  and  the  primary  Southeast  Regional  Distribution 
Point  for  all  Department  of  Defense  (DOD)  munitions”  (BGAD,  n.d.)  Sitting  on  over  14,000 
acres  and  possessing  a  storage  capacity  of  over  3  million  square  feet,  it  supports  the  DoD 
through  the  receipt,  storage,  maintenance,  shipping,  and  demilitarization  of  a  vast  variety  of 
standard  and  non-standard  ammunition  (BGAD,  n.d.). 

2.  Letterkenny  Munitions  Center 

Letterkenny  Munitions  Center  is  a  tenant  of  Letterkenny  Army  Depot  and  located  in 
Chambersburg,  Pennsylvania.  Occupying  about  16,000  acres,  it’s  a  Strategic  Mobility  Platform, 
specializing  in  the  receipt,  storage,  and  maintenance  of  a  variety  of  Army,  Air  Lorce,  and  Navy 
missiles  systems.  These  systems  include  the  Sidewinder,  Sparrow,  High-speed  Anti-radiation 
Missile  (HARM),  Joint  Air-to-Surface  Stand-off  Missile  (JASSM),  and  the  Advanced  Medium 
Range  Air-to-Air  Missile  (AMRAAM;  Crane  Army  Ammunition  Activity,  n.d.-b). 
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3.  McAlester  Army  Ammunition  Plant 

Located  in  McAlester,  Oklahoma,  McAlester  Army  Ammunition  Plant  (MCAAP)  is  the 
DoD’s  premier  facility  for  loading  bombs  with  energetics  such  as  TNT.  Covering  44,964  acres 
and  with  a  storage  capacity  of  over  8.8  million  square  feet,  it  shares  many  of  the  capabilities  of 
BGAD  in  that  it  receives  stores,  maintains,  ships  and  demilitarizes  a  huge  variety  of  munitions 
(MCAAP,  n.d.). 

4.  Tooele  Army  Depot 

Tooele  Army  Depot  is  located  in  Tooele,  Utah.  Occupying  23,610  acres  and  possessing 
more  than  2.4  million  square  feet  of  storage  space,  its  mission  is  similar  to  that  of  BGAD  and 
MCAAP  in  that  it  receives,  stores,  maintains,  ships,  and  demilitarizes  munitions.  But  unlike  the 
other  components  it  also  designs,  manufactures,  and  supports  the  special  equipment  needed  to 
perform  ammunition  maintenance  and  demilitarization  activities  (Tooele  Army  Depot,  n.d.). 

5,  Crane  Army  Ammunition  Activity 

As  a  tenant  of  the  Navy  Region  Midwest,  Naval  Support  Activity,  Crane  Army 
Ammunition  Activity  is  located  in  Crane,  Indiana.  It  occupies  more  than  51,000  acres  and  can 
store  650,000  tons  of  ammunition  related  stock.  Like  BGAD  and  MCAAP,  its  primary  mission 
is  to  receive,  store,  ship,  produce,  renovate,  and  demilitarize  conventional  ammunition  (Crane 
Army  Ammunition  Activity,  n.d.-a). 

6,  Pine  Bluff  Arsenal 

Located  in  Pine  Bluff,  Arkansas,  Pine  Bluff  Arsenal’s  primary  mission  is  the  production 
of  smoke,  incendiary,  and  pyrotechnic  munitions  and  devices.  It  also  tests  chemical  defense 
clothing  (Bearden,  2012). 

G.  BILLS  OF  MATERIAL  DATA  QUALITY  AUDIT  PROCESS  DESCRIPTION 

As  the  aforementioned  components  of  the  JMC  execute  their  assigned  missions,  they 
have  the  responsibility  of  achieving  and  maintaining  data  quality  goals,  as  well  as  performing 
data  quality  audits.  However,  the  management  of  the  overall  data  quality  program  belongs  to  the 
JMC  Enterprise  Integration  Data  Team. 


13 


Before  an  audit  begins,  the  JMC  Enterprise  Integration  Data  Team  notifies  the  BOM 
business  proeess  owner  (BPO)  of  eaeh  eomponent  organization  and  sends  a  standardized  JMC 
data  aeeuraey  seoreeard  with  whieh  to  eonduet  the  audit.  Currently,  the  seoreeard  is  a 
customized  spreadsheet. 

During  the  first  phase  of  the  audit,  the  BPOs  check  each  of  the  LMP  BOM  records 
according  to  the  completeness  and  accuracy  of  13  data  elements.  These  elements  consist  of 
BOM  usage,  base  quantity,  valid-from  date,  item  category,  BOM  component,  component 
quantity,  component  unit  of  measure,  explosive  type,  inductive  recursiveness  allowed,  inductive 
relevancy  to  costing,  issue  location,  component  supply  area,  and  special  process  type.  During 
the  audit,  records  found  with  at  least  one  incorrect  element  are  designated  “fail.”  Alternatively, 
if  all  elements  of  a  record  are  determined  to  be  correct,  the  record  receives  a  “pass.” 

As  the  first  phase  is  being  completed,  the  scorecard  automatically  populates  a  data 
summary  that  displays  critical  metrics  such  as  audit  date,  total  BOMs  audited,  total  BOMs 
passed,  total  BOMs  failed,  percent  accuracy,  percent  accuracy  target,  defect  quantities  by  critical 
data  element,  total  defects,  and  defects  per  defective  record. 

During  the  second  phase,  the  BPO  and  subordinate  personnel  identify  the  root  causes  of 
the  failures  as  well  as  corrective  actions  that  should  prevent  failure  reoccurrence.  Both  failures 
and  corrective  actions  are  then  recorded  directly  into  the  scorecard. 

Once  the  audit  is  complete,  the  BOM  records  and  audit  findings  are  submitted  to  the  JMC 
Enterprise  Integration  Data  Team  for  validation.  After  the  results  are  validated,  the  scorecards 
are  posted  on  the  JMC  Enterprise  Integration  Data  Team’s  SharePoint  site  and  retained  by  the 
team  to  use  on  subsequent  audits  (R.  Puller,  personal  communication,  June  19,  2012). 

H.  SUMMARY 

In  this  chapter  the  author  began  with  a  literature  review  that  included  a  discussion  on 
innovative  training  strategies.  That  discussion  stemmed  from  a  2010  GAO  report  that  was  critical 
of  how  the  Army  trained  its  personnel  for  using  the  EMP.  Afterwards,  the  author  discussed  the 
development  of  the  EMP,  the  nature  of  data  quality,  and  the  missions  of  JMC  component 
organizations.  The  author  wrapped  up  the  chapter  with  an  overview  of  the  data  audit  process.  In 
Chapter  III,  the  author  displays  BOM  defect  data  from  six  major  components  of  the  JMC. 
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III.  BOM  DEFECT  DATA  SUMMARY 


In  order  to  respect  confidentiality,  direct  references  to  specific  organizations  have 
been  changed  to  the  generalized  names  of  Component  A,  Component  B,  and  so  forth. 
The  defect  data  in  Tables  2-7  originated  from  scorecards  published  by  the  JMC 
Enterprise  Integration  Data  Team  (R.  Fuller,  personal  communication,  June  19,  2012). 
Tables  8-13  and  Figures  4-10  were  created  using  data  from  the  same  scorecards.  The 
author  added  the  qualification  data  in  the  table  columns  labeled  “category.”  The 
qualification  data  was  also  used  in  the  figures.  For  the  sake  of  authenticity,  error  and  root 
cause  descriptions  were  taken  verbatim  from  the  scorecards. 
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A, 


SCORECARD  DEFECT  DATA 


Table  2.  Defect  Data  for  Component  A  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 

DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Audit  Date 

BOMs 

Audited 

Totai 

BOMs 

Passed 

Total 

BOMs 

Failed 

% 

Accuracy 

Target  % 
Accuracy 

BOM  usage 

base  quantity 

valid  from-date 

item  category 

BOM  component 

component  quantity 

component  unit  of  measure 

explosive  type 

inductive  recursiveness  allowed 

inductive  relevancy  to  costing 

issue  location 

component  supply  area 

special  process  type 

Total 

Defects 

Defects 

per 

Defective 

Record 

9/7/2011 

370 

73 

297 

19.7% 

0 

o 

1 

o 

0 

1,245 

4.2 

Note.  The  data  in  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
eommunieation,  June  19,  2012). 
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Table  3.  Defect  Data  for  Component  B  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 

DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Audit  Date 

BOMs 

Audited 

Total 

BOMs 

Passed 

Total 

BOMs 

Failed 

% 

Accuracy 

Target  % 
Accurac 

y 

BOM  usage 

base  quantity 

valid  from-date 

item  category 

BOM  component 

component  quantity 

component  unit  of  measure 

explosive  type 

inductive  recursiveness  allowed 

inductive  relevancy  to  costing 

issue  location 

component  supply  area 

special  process  type 

Total 

Defects 

Defects 

per 

Defectiv 
e  Record 

2/2/2012 

133 

99 

34 

74.4% 

0 

0 

0 

4 

0 

0  0  0  137 

4 

9/26/201 1 

155 

58 

97 

37.4% 

0 

0 

0 

0 

0 

0  0 

452 

4.7 

Note.  The  data  in  this  table  was  taken  from  scoreeards  the  author  was  given  aecess  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
eommunieation,  June  19,  2012). 
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Table  4.  Defect  Data  for  Component  C  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 


Audit  Date 


3/29/2012 


2/2/2012 


9/27/201 1 


9/13/2011 


BOMs 

Audited 


247 


210 


199 


199 


Total 

BOMs 

Passed 


246 


194 


177 


199 


Total 

BOMs 

Failed 


16 


22 


% 

Accuracy 


99.6% 


92.4% 


88.9% 


100% 


Target  % 
Accurac 
J 


98.0% 


98.0% 


98.0% 


98.0% 


DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 


10 


11 


12 


13 


Total 

Defects 


84 


117 


Defects 

per 

Defectiv 
e  Record 


5.3 


5.3 


Note.  The  data  in  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
communication,  June  19,  2012). 
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Table  5.  Defect  Data  for  Component  D  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 

DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Audit  Date 

BOMs 

Audited 

Total 

BOMs 

Passed 

Total 

BOMs 

Failed 

% 

Accuracy 

Target  % 
Accurac 

y 

BOM  usage 

base  quantity 

valid  from-date 

item  category 

BOM  component 

component  quantity 

component  unit  of  measure 

explosive  type 

inductive  recursiveness  allowed 

inductive  relevancy  to  costing 

issue  location 

component  supply  area 

special  process  type 

Total 

Defects 

Defects 

per 

Defectiv 
e  Record 

5/24/2012 

135 

131 

4 

97.0% 

98.0% 

0 

0 

0 

0 

0 

0 

0 

0 

0 

o 

1 

o 

0 

4 

1.0 

1/11/2012 

131 

130 

4 

99.2% 

98.0% 

0 

0 

0 

0 

0  0 

0 

0 

0 

0 

0 

0 

1 

1.0 

9/13/2011 

195 

183 

12 

93.8% 

98.0% 

0 

0 

0 

0 

0 

o 

1 

o 

1 

o 

1 

o 

24 

2.0 

Note.  The  data  in  this  table  was  taken  from  scoreeards  the  author  was  given  aecess  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
eommunieation,  June  19,  2012). 
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Table  6.  Defect  Data  for  Component  E  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 

DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Audit  Date 

BOMs 

Audited 

Totai 

BOMs 

Passed 

Total 

BOMs 

Failed 

% 

Accuracy 

Target  % 
Accuracy 

BOM  usage 

base  quantity 

valid  from-date 

item  category 

BOM  component 

component  quantity 

component  unit  of  measure 

explosive  type 

inductive  recursiveness  allowed 

inductive  relevancy  to  costing 

issue  location 

component  supply  area 

special  process  type 

Total 

Defects 

Defects 

per 

Defective 

Record 

10/24/2011 

85 

58 

27 

68.2% 

81 

3 

Note.  The  data  in  this  table  was  taken  from  scorecards  the  author  was  given  access  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
communication,  June  19,  2012). 
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Table  7.  Defect  Data  for  Component  F  (From;  R.  Fuller,  personal  communication,  June  19,  2012) 


DATA  OBJECT  ACCURACY  SUMMARY 

DEFECT  QUANTITIES  BY  CRITICAL  DATA  ELEMENT 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Audit  Date 

BOMs 

Audited 

Total 

BOMs 

Passed 

Total 

BOMs 

Failed 

% 

Accuracy 

Target  % 
Accuracy 

BOM  usage 

base  quantity 

valid  from-date 

Item  category 

BOM  component 

component  quantity 

component  unit  of  measure 

explosive  type 

Inductive  recursiveness  allowed 

Inductive  relevancy  to  costing 

issue  location 

component  supply  area 

special  process  type 

Total 

Defects 

Defects 

per 

Defective 

Record 

11/22/2011 

123 

123 

4 

100.0% 

98.0% 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0.0 

11/4/2011 

741 

740 

4 

99.9% 

98.0% 

0 

0 

0 

0  0 

0 

0 

0 

0 

0 

0 

0 

2 

2.0 

9/30/201 1 

741 

168 

573 

22.7% 

98.0% 

o 

1 

o 

1 

o 

1 

o 

0  0 

3.7 

Note.  The  data  in  this  table  was  taken  from  scorecards  the  author  was  given  access  to  on  the  JMC  Data  Quality  Team's  SharePoint  website  (R.  Fuller,  personal 
communication,  June  19,  2012). 
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B,  DEFECT  DATA  QUALIFIED  BY  CATEGORY 


Table  8.  Defect  Qualification  Data  for  Component  A  (After:  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit 

Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

9/7/2011 

BOM  Usage 

BOM  Usage  cCode  "R" 

Change  in  BOM  business  rules  after  originally  built 

Policy 

46 

9/7/2011 

Item  Category 

Code  "Y" 

Personnel  was  not  fully  educated  at  time  of  creation  of  BOMs. 
JM&L  BOM  process  has  changed  since  go-live.  Should  have 
been  "L"  or "N" 

Training 

45 

9/7/2011 

Explosive  Type 

Blank  or  incorrect  type 

Change  in  business  rules  and  not  understanding  how  each  tyoe 
worked  with  Planning  MRP/Inventory 

Training 

141 

9/7/2011 

Ind-Recursiveness- 
Al  lowed 

Entered  wrong  data 

Did  not  properly  mark  Demil  component  for  recursiveness 

Training 

1 

9/7/2011 

Ind-Relevancy-to-Costing 

Entered  incorrect  data  or  left  blank 

No  standard  business  rule  for  relevancy  costing  at  time  of 
creation 

Policy 

9 

9/7/2011 

Issue-Location 

Storage  Location  Left  Blank 

Personnel  did  not  enter  storage  location 

Training 

101 

9/7/2011 

Component-Supply-Area 

Left  Blank 

Personnel  did  not  enter  Supply  Area 

Training 

899 

9/7/2011 

Special-Proc-Type 

Incorrect  data 

Lack  of  knowledge  when  creating  BOM 

Training 

3 

Note.  The  data  in  the  first  four  eolumns  of  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  eommunieation  June  19,  2012). 
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Table  9.  Defect  Qualification  Data  for  Component  B  (After:  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit  Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

9/26/2011 

Component-Supply-Area 

Supply  Area  Blank  or  incorrect 

Inadequate  guidance,  was  not  initially  required  to  be 
populated 

Training 

190 

9/26/2011 

BOM-Usage 

Actually  Incorrect  BOM  Status 

Inactive  BOMs  that  were  marked  as  active,  human 
error 

Training 

165 

9/26/2011 

Explosion-Type 

Left  Blank 

Operator  error,  not  marked  when  making  the  BOM 

Training 

39 

9/26/2011 

Ind-Recursiveness- 

Allowed 

(5)  Incorrectly  flagged  as  Recursive-Allowed 

Initial  setup  not  changed,  guidance 

Training 

5 

9/26/2011 

Ind-Recursiveness- 

Allowed 

(16)  Flagged  as  recursive,  if  not  causes  error 
in  production  results 

Business  process  changed,  recursive  required  for 
correct  production  results,  does  not  fit  with  business 
rules 

Policy 

16 

9/26/2011 

Ind-Relevancy-to-Costing 

Incorrectly  flagged  for  Costing 

Setup  according  to  initial  guidance  not  updated, 
business  process  changed  not  updated  during  change 

Policy 

21 

9/26/2011 

Issue-Location 

Left  Blank 

Changed  business  process,  not  updated  during  change 

Policy 

16 

2/2/2012 

Component  Supply  Area 

Supply  Area  Blank  or  incorrect  on  BOMs 

Guidance  received  included  placing  PSA  on  work 
centers,  this  does  not  show  on  the  BOM  audits,  per 
JM&L  guidance  not  to  be  counted  as  a  BOM  audit  error 

Policy 

8 

2/2/2012 

Ind-Relevancy-to-Costing 

Incorrectly  identified  for  Costing 

Based  on  MRP  running  and  changes  resulting  from 
eliminating  recursiveness,  not  all  BOM  components  had 
been  updated 

Process 

59 

2/2/2012 

Issue-Location 

Incorrect  value  or  left  blank 

WH  location  not  reviewed  and  added  or  updated  while 
preparing  for  MRP 

Training 

30 

2/2/2012 

Explosion-Type 

Left  Blank 

Operator  error,  not  marked  when  making  the  BOM 

Training 

17 

2/2/2012 

Special  Procument  Type 

Incorrect  value  or  left  blank 

As  a  result  of  recursive  changes  and  MRP  being  run, 
BOMs  were  revised,  not  all  corrections  made 

Process 

14 

2/2/2012 

BOM-Usage 

Actually  Incorrect  BOM  Status 

As  a  result  of  recursive  changes  and  MRP  being  run, 
BOMs  were  not  revised  to  be  inactive,  human  error 

Process 

13 

2/2/2012 

BOM-Component 

Components  not  deleted  from  BOM 

As  a  result  of  recursive  changes  and  MRP  being  run, 
BOMs  were  changed  and  components  were  not 
deleted  from  the  BOM,  human  error 

Process 

4 

Note'.  The  data  in  the  first  four  eolumns  of  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  eommunieation  June  19,  2012). 
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Table  10.  Defect  Qualification  Data  for  Component  C  (After;  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit 

Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

9/27/2011 

Component  supply 

area 

should  be  blank  for  all  negative  quantities 

error  due  to  new/updated  JM&L  business  rules 

Policy 

40 

9/27/2011 

relevancy  to  costing 

Not  cost  relevant  for  negative  quantities 

error  due  to  new/updated  JM&L  business  rules 

Policy 

31 

9/27/2011 

explosion  type 

should  be  marked  as  R2 

error  due  to  new/updated  JM&L  business  rules 

Policy 

27 

9/27/2011 

issue  location 

blanks  not  allowed 

error  due  to  new/updated  JM&L  business  rules 

Policy 

16 

9/27/2011 

recursiveness  allowed 

recursiveness  not  allowed  for  negative 
quantities 

error  due  to  new/updated  JM&L  business  rules 

Policy 

3 

2/2/2012 

explosion  type 

should  be  marked  as  R2  for  non-text 
components 

human  error 

Training 

7 

2/2/2012 

relevancy  to  costing 

Not  cost  relevant  for  negative  quantities 

human  error 

Training 

49 

2/2/2012 

issue  location 

blanks  not  allowed  for  non-text  components 

human  error 

Training 

7 

2/2/2012 

supply  area 

should  be  blank  for  all  negative  quantities 

human  error 

Training 

21 

3/29/2012 

relevancy  to  costing 

Not  cost  relevant  for  negative  quantities 

human  error 

Training 

3 

Note'.  The  data  in  the  first  four  eolumns  of  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  eommunieation  June  19,  2012). 
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Table  11.  Defect  Qualification  Data  for  Component  D  (After:  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit 

Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

9/13/2011 

Component  Supply  Area 

Incorrect  supply  area 

Definition  was  changed  during  the  audit; 
Data  was  correct  at  time  of  audit  but  not 
at  time  of  data  pull 

Policy 

16 

9/13/2011 

Costing 

Does  not  show  costing  relevancy 

Human  error;  planner  entered  incorrect 
data  when  building  BOM 

Training 

4 

9/13/2011 

Unit  of  Measure 

Material  master  error 

Cataloging  error  in  legacy  data 

Process 

3 

9/13/2011 

Explosive  Type 

Incorrect  explosive  type 

Explanation  of  difference  between  R1 
and  R2  was  unclear,  resulting  in  error 
when  building  BOM 

Training 

1 

1/11/2012 

Component  -  Quantity 

Value  listed  as  a  positive  number  instead 
of  negative  number 

Making  corrections  to  the  unit  of 
measure  in  the  baseline  audit  caused  the 
change  in  the  component  quantity  critical 
element 

Training 

1 

Note'.  The  data  in  the  first  four  columns  of  this  table  was  taken  from  scorecards  the  author  was  given  access  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  communication  June  19,  2012). 
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Table  12.  Defect  Qualification  Data  for  Component  E  (After:  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit  Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

10/24/2011 

BOM  Component 

Components  were  deleted  or  added 

BOM  changes  were  not  communicated  to  the  LMP 
workers 

Process 

40 

10/24/2011 

Component  Qty 

Qty  of  tape  was  fine-tuned  or  usage  factors  were 
inserted  into  RBOMs  (TACOM  workload) 

Paying  closer  attention  to  processes  but  not 
communicating  to  the  LMP  workers 

Training 

27 

10/24/2011 

Issue  Location 

Blank  when  should  not  have  been 

Requirement  to  have  this  set,  not  widely 
communicated 

Process 

1 

10/24/2011 

BOM  Usage 

Component  was  deleted  so  SME  marked  all  fields 

Ambiguous  issue,  should  all  be  marked  F  if 
component  is  deleted  or  added;  regular  DO  lead 
gone  for  1  wk  and  fill-in  not  familiar  with  audits 
(many  hats,  less  heads) 

Training 

1 

10/24/2011 

Base  Qty 

Component  was  deleted  so  SME  marked  all  fields 

Training 

1 

10/24/2011 

Valid  from  Date 

Component  was  deleted  so  SME  marked  all  fields 

Training 

1 

10/24/2011 

Item  Category 

Component  was  deleted  so  SME  marked  all  fields 

Training 

1 

10/24/2011 

C-UOM 

Component  was  deleted  so  SME  marked  all  fields 

Training 

1 

10/24/2011 

Explosion  Type 

Component  was  deleted  so  SME  marked  all  fields 

Training 

1 

Note'.  The  data  in  the  first  four  columns  of  this  table  was  taken  from  scorecards  the  author  was  given  access  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  communication  June  19,  2012). 
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Table  13.  Defect  Qualification  Data  for  Component  F  (After:  R.  Fuller,  personal  communication,  June  19,  2012) 


Audit 

Date 

Data  Element 

Error  Description 

Root  Cause 

Category 

Qty 

9/30/2011 

Base  Quantity 

Initial  input  was  for  total  number  of  parts  generated 
from  a  bar  of  aluminum  for  a  grenade  tops  and 
bottoms  project 

Input  was  from  Qct  1, 2009;  initial  learning  phase 

Training 

2 

9/30/2011 

Item  Category 

N  entered;  Should  be  L 

N  Non-Stock  L  Stock  Incorrect  input.  Confusion  on 
what  should  be  entered.  All  errors  are  basically 
from  2  BQM's  and  2  dates.  This  error  was  noted 
prior  to  audit  because  of  issues  related  to  MRP. 

Training 

90 

9/30/2011 

BOM  Component 

Wrong  Part  Number  entered 

Typographical  error 

Process 

2 

9/30/2011 

ExplosionType 

Missing  ExplosiveType.  Should  read  R2 

Majority  of  errors  from  5  Bombs  and  5  dates;  lack 
of  user  knowledge 

Training 

200 

9/30/2011 

Issue  Storage 

Location 

Missing  Issue  Location.  Should  read  WH01  or 
WH05 

Majority  of  errors  were  conducted  on  5  dates;  lack 
of  user  knowledge 

Training 

249 

9/30/2011 

Supply  Area 

Missing  Component  Supply  Area.  For  example 
should  read  059400-05B  or  059400-01 B 

This  seems  to  be  a  systemic  problem.  This  was  a 
late  requirement  that  was  handed  down  in  late 
August/September  2010.  Many  BQMs  were  built 
prior  to  guidance  and  have  not  been  changed.  Also 
issue  with  user  knowledge. 

Policy 

1577 

9/30/2011 

Component  Quantity 

Incorrect  Quantity  for  what  is  required 

Typographical  error 

Process 

5 

11/4/2011 

BOM  Component 

Pending  Material  Master  typographical  error 
confirmation.  Possible  data  pull  anomaly.  Actual 
BQM  reflects  correct  material  numbers. 

Process 

2 

Note'.  The  data  in  the  first  four  eolumns  of  this  table  was  taken  from  seoreeards  the  author  was  given  aeeess  to  on  the  JMC  Data  Quality  Team's  SharePoint 
website  (R.  Fuller,  personal  eommunieation  June  19,  2012). 
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Figure  1 .  Component  A  Defects  by  Category 


Figure  2.  Component  B  Defects  by  Category 
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Figure  3.  Component  C  Defects  by  Category 
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Figure  4.  Component  D  Defects  by  Category 
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Figure  5.  Component  E  Defects  by  Category 
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Figure  6.  Component  F  Defects  by  Category 
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Figure  7.  Overall  Defects  by  Category 
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IV.  ANALYSIS 


A.  DATA  QUALIFICATION 

The  data  in  Tables  2-7  comes  directly  from  scorecards  generated  by  the 
component  organizations  of  the  JMC.  The  benefit  of  these  tables  is  that  they  allow  the 
reader  to  see  the  types  and  number  of  defects  for  each  audit  that  was  conducted.  For 
example,  in  Table  3  the  data  shows  that  on  September  26,  2011,  Component  B  conducted 
an  audit  of  155  BOMs.  Within  those  BOMs,  a  total  of  452  defects  were  found.  Of  those 
defects,  none  were  with  the  special  process  data  element;  however,  there  were  190 
defects  associated  with  the  component  supply  data  element.  At  the  next  audit  on 
February  2,  2012,  133  BOMs  were  audited  and  a  total  of  137  defects  were  found.  Of 
those  defects,  14  were  special  process  type  defects  and  there  were  no  component  supply 
defects. 

Based  on  these  observations,  it  is  evident  that  although  the  total  number  of  defects 
dropped  at  the  second  audit,  the  reduction  did  not  extend  consistently  to  all  the  data 
elements.  For  example,  the  special  process  type  defects  actually  increased  by  14,  while 
the  component  supply  defects  dropped  from  190  to  0.  It  is  clear  that  some  degree  of 
defect  qualification  is  needed  to  obtain  anything  meaningful  from  this  data. 

Tables  8-13  incorporate  BPO  input  in  the  error  description  and  root  cause 
columns  that  introduce  the  first  level  of  qualification.  The  information  is  particularly 
useful  in  that  it  provides  tactical-level  reasons  why  defects  occurred.  For  example,  the 
data  in  Table  3  shows  that  there  were  39  explosive-type  defects  from  the  September  26 
audit  of  Component  B,  but  in  the  error  description  and  root  cause  columns  of  Table  9,  the 
data  indicates  that  those  defects  were  due  to  blank  data  fields  that  were  not  marked  as  the 
BOM  was  being  created. 

Possessing  the  degree  of  detail  shown  in  Tables  8-13  is  helpful  in  identifying 
defect  causes.  However,  the  specificity  of  the  defect  causes  hindered  the  author  from 
performing  a  meaningful  analysis.  The  author  also  reasoned  that  the  development  of  a 
strategy  for  eventually  reducing  defects  would  require  a  higher  level  of  qualification.  It 
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was  therefore  neeessary  to  further  qualify  the  data  by  grouping  the  root  eauses  into 
deseriptive  eategories.  In  surveying  the  root  eauses,  the  author  observed  that  they  eould 
be  grouped  into  three  categories:  training,  process,  and  policy.  For  example,  the 
aforementioned  39  explosive-type  defects  that  had  unmarked  data  fields  were  categorized 
as  training  because  JMC  personnel  presumably  did  not  know  the  required  documentation 
procedure  and  lacked  adequate  training.  In  another  instance  on  Table  9,  the  13  BOM 
usage  defects  were  categorized  as  process  because  the  root  cause  information  showed 
that  the  defects  could  have  been  prevented  had  the  BPO  implemented  a  procedure 
requiring  the  removal  of  obsolete  components  whenever  the  MRP  was  implemented.  In 
still  another  example,  21  inductive  relevancy  to  costing  defects  were  categorized  as 
policy  because  the  root  cause  description  indicated  a  level  of  uncertainty  regarding  the 
application  of  business  rules.  Although  this  categorization  method  is  not  an  exact 
science,  the  author  believes  that  enough  information  is  present  in  the  data  to  adequately 
delineate  between  the  categories  without  introducing  significant  amounts  of  subjectivity. 

B,  DATA  TRENDS 

With  the  data  qualified  into  three  different  categories,  seeing  the  trends  and 
patterns  is  a  much  more  obvious  exercise.  Figures  4-9  show  the  qualified  defects  by 
quantity  and  the  audit  dates  for  each  of  the  six  components  of  the  JMC,  and  Figure  10 
gives  a  similar  but  more  cumulative  look  at  data  defects. 

1,  Training 

As  was  no  surprise,  inadequate  training  was  the  greatest  overall  source  of  data 
defects,  as  seen  in  Figure  10.  The  2010  GAO  report  referenced  in  Chapter  II  linked 
training  with  data  quality.  However,  it  was  interesting  to  notice  how  quickly  the  training 
defects  decreased  with  subsequent  audits.  The  only  exception  to  this  trend  of  decreasing 
defects  was  observed  in  Figure  6  with  Component  C.  This  indicates  that  significant 
learning  may  be  happening  through  the  auditing  process  rather  than  through  the  training 
processes.  While  it  is  certainly  commendable  to  learn  from  the  audits,  the  audit  process 
is  designed  to  gain  status  and  not  provide  training.  Because  of  this,  the  declining  trend  in 
training-related  data  defects  may  not  be  an  indicator  that  the  actual  training  has  improved. 
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2,  Policy 

Policy-related  defects  were  surprisingly  the  seeond  highest  souree  of  data  defeets, 
not  far  behind  training-related  defects.  In  Components  C,  D,  and  F,  policy  was  actually 
the  leading  cause  of  defeets.  Like  training,  policy-related  defects  tended  to  deerease  with 
subsequent  audits.  This  is  indieative  of  eomponent  organizations  that  initially  had 
shifting,  non-existent,  or  amorphous  policies  concerning  the  management  of  BOM  data 
within  the  LMP.  At  some  later  point,  perhaps  in  part  due  to  audits,  these  organizations 
developed  workable  policies.  This  point  is  espeeially  clear  in  Table  13,  whieh  shows  that 
1,577  policy  defects  were  attributed  to  a  policy  change  that  apparently  oecurred  after  the 
BOMs  were  already  built. 

3,  Process 

The  author  anticipated  that  proeess-related  issues  were  a  significant  source  of 
defects.  In  Component  E,  proeess  defects  were  in  faet  the  largest  source  of  data  defects. 
However,  in  all  the  other  components  defects  directly  related  to  proeess  issues  were 
virtually  non-faetors.  Considering  that  the  sample  size  for  Component  E  was  relatively 
small,  the  high  number  of  proeess  defeets  observed  could  easily  have  been  an  outlier. 

Although  the  author  did  not  observe  a  direet  eonnection  between  proeess  and  data 
defeets,  the  analysis  does  indieate  an  indirect  relationship  by  way  of  training  and  policy 
issues.  Study  of  the  training  and  policy  defect  data  indicates  that  improving  the  internal 
processes  inherent  to  training  and  policy  development  can  present  opportunities  to  reduce 
data  defeets.  Eor  example,  in  Table  13  the  541  training  defeets  of  Component  E  could 
possibly  have  been  avoided  if  Component  E  had  implemented  a  process  for  assessing  and 
addressing  the  knowledge  gaps  of  each  user  providing  inputs  to  BOM  data.  A  similar 
point  can  be  made  regarding  the  need  for  a  proeess  that  synthesizes  the  best  data 
management  policies.  Additionally,  there  is  a  need  for  a  better  proeess  of  updating  and 
communicating  policy  changes  as  conditions  warrant. 
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c. 


APPLICATION  WITHIN  THE  LMP 


Up  to  this  point,  the  discussion  has  centered  primarily  around  data  quality  within 
the  LMP.  By  focusing  on  such  a  narrow  topic,  the  bigger  picture  of  data  quality’s 
significance  within  the  LMP  should  be  clarified.  The  life  blood  of  any  AIS  is  the 
information  within  it.  No  matter  how  impressive  the  processing  power,  memory, 
bandwidth,  or  any  other  system  specification,  the  system  as  a  whole  is  of  little  value 
without  data.  To  take  this  point  a  step  further,  even  if  a  highly  capable  AIS  has  data,  the 
system  is  still  of  little  value  unless  the  data  is  of  good  quality — meaning  that  it  is 
accurate,  accessible,  believable,  useful,  and  so  forth.  As  the  Army’s  latest  major  AIS,  the 
LMP  is  utilized  by  the  JMC  to  manage  over  $30  billion  in  weaponry,  as  well  as  the 
supporting  structure  of  multiple  thousands  of  people  and  facilities  spread  across  several 
states — a  mission  that  is  desperately  dependent  upon  quality  data. 

An  example  will  help  to  illustrate  the  link  between  data  quality  and  a  typical  JMC 
operation.  The  105mm  round  is  a  popular  munition  used  by  the  Army,  Air  Force,  and 
Marines.  The  MCAAP  is  one  of  a  few  components  of  the  JMC  that  stores  tens  of 
thousands  of  105mm  rounds.  In  this  example,  the  JMC  provided  funds  to  the  MCAAP  to 
renovate  a  special  variant  of  the  105mm  round  that  was  urgently  needed  to  support 
operations  in  Afghanistan.  A  crucial  part  of  the  renovation  process  is  replacing  non- 
serviceable  105mm  round  components.  These  components  are  listed  on  a  105mm  round 
BOM.  One  of  those  components  that  typically  needs  replacement  is  the  fuze.  In  this 
example,  an  ammunition  planner — ^who  is  also  an  LMP  user — mistakenly  overwrites  the 
unit  of  measure  element  for  the  fuze  on  the  BOM  as  pallets  whereas  it  should  be  as  each. 
In  doing  so,  the  ammunition  planner  unknowingly  created  defective  data.  This  defective 
BOM  data  was  passed  along  to  other  LMP  users,  such  as  to  the  procurement  personnel 
who  order  supplies.  In  this  case,  due  to  the  defective  unit  of  measure  used  for  the  fuze, 
the  procurement  personnel  failed  to  order  enough  fuzes  to  service  all  the  rounds  the 
MCAAP  was  funded  to  perform.  The  immediate  consequence  was  that  the  105mm 
renovation  line  came  to  a  halt  once  the  fuzes  run  out  and  did  not  restart  until  more  fuzes 
were  ordered,  produced,  shipped,  delivered,  and  received — a  process  that  takes  about  a 
month.  In  turn,  this  resulted  in  either  the  Afghanistan  mission  being  delayed  by  a  month, 
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or  in  the  warfighters  having  to  resort  to  riskier  eontingency  plans  to  carry  out  their 
missions,  which  increased  the  likelihood  of  mission  failure  and  casualties.  In  this 
business,  data  quality  matters. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


In  conclusion,  this  study  provides  insight  into  how  the  JMC  can  significantly 
improve  data  quality  in  its  eomponent  organizations.  Based  on  the  data  analyzed  in  this 
thesis,  there  must  be  a  strategie  foeus  on  improving  training  and  policy  formulation. 
Furthermore,  the  JMC  must  invest  effort  into  the  development  of  solid  internal  proeesses 
for  supporting  training  and  poliey  management  sinee  proeesses  appear  to  have  an  indireet 
impaet  on  data  quality  within  the  LMP. 

A,  TRAINING  RECOMMENDATIONS 

As  this  study  has  shown,  the  training  approaeh  provided  by  the  Army  was 
inadequate.  The  faet  that  subsequent  audits  showed  signifieant  reduetions  in  data  defeets 
suggests  that  alternative  forms  of  training  must  have  oeeurred  within  the  various 
eomponents  of  the  JMC.  It  is  therefore  reeommended  that  the  Army  adopt  learner-eentrie 
training  strategies  and  utilize  them  to  edueate  new  users  of  the  LMP,  as  well  as  to  meet 
the  training  needs  of  users  with  varying  levels  of  experienee.  The  training  program 
should  be  fully  eompliant  with  the  concepts  of  the  ALC  2015  (TRADOC,  2011)  strategy 
that  was  discussed  in  Chapter  II. 

B,  POLICY  RECOMMENDATIONS 

One  particularly  troublesome  aspect  of  policy  defects  is  that  no  matter  how  well 
the  users  are  trained,  a  shifting  policy,  a  poor  policy,  or  a  formerly  good  poliey  that  no 
longer  serves  the  interests  of  the  mission  based  on  ehanging  eonditions  ean  have  a  hugely 
negative  impaet  on  data  quality.  Based  on  the  findings  of  this  study,  the  JMC  must  be 
vigilant  in  defining,  implementing,  eommunieating,  and  maintaining  its  polieies  related  to 
the  usage  of  the  LMP.  With  so  many  eomponents  and  subeomponents,  it  would  be  very 
easy  for  the  JMC  to  beeome  paroehialized  regarding  the  LMP.  A  stovepiped  approaeh 
would  undermine  its  effectiveness. 
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C.  PROCESS  RECOMMENDATIONS 

In  order  to  carry  out  the  training  and  policy  recommendations  they  must  each  be 
undergirded  with  solid  processes.  The  training  processes  that  the  JMC  should  develop 
must  be  apt  at  assessing  the  training  needs  of  the  LMP  users  and  systematically 
delivering  them  knowledge  that  is  both  accurate  and  pertinent  to  their  jobs  on  a  timely 
basis  and  in  a  manner  that  adapts  to  the  individual  user’s  learning  capability.  Likewise, 
the  JMC  should  create  processes  that  support  sound  policy  management.  Policy 
processes  should  require  the  thorough  review  of  the  impacts  of  adopting  a  proposed 
policy  as  well  as  the  consequences  of  rejecting  it.  Processes  that  promote  comprehensive 
policy  reviews  should  minimize  the  need  for  policy  changes.  However,  when  the 
inevitable  need  for  a  change  occurs,  there  should  be  a  policy  change  control  process  for 
ensuring  that  the  change  is  executed  in  a  manner  that  does  not  create  defective  data. 

D,  FINAL  THOUGHTS 

As  more  audits  are  conducted,  the  author  hopes  that  this  study  will  encourage  an 
effort  to  discover  more  opportunities  for  reducing  data  defects  within  the  LMP.  It  is 
possible  that  three  categories  of  qualification  is  too  simplistic  and  that  more  defect 
categories  are  necessary.  As  more  audit  data  is  captured,  and  more  analysis  is  conducted, 
time  will  tell. 

On  a  final  note,  the  LMP  is  the  system  by  which  the  JMC  conducts  its  business. 
Because  of  the  potential  downstream  effects  in  terms  of  money  and  human  lives,  the  data 
quality  within  this  system  is  profoundly  important.  By  drawing  attention  to  the 
importance  of  data  quality,  this  study  makes  it  possible  for  the  Army  to  improve  its 
support  of  the  warfighter. 
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APPENDIX 


Figure  8. 
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Blended  Learning  Environment  (From;  Plifka,  201 1,  p.  23) 
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Some  Characteristics  of  a 
Learner-Centric  2015  Learning  Environment 


Single  Portal  to  Digital 
Learning  Resources 


Virtual  Training 
Environments 

(e  g  JTCOIC-Training  Brain) 


Soldier  Created  Content 

(Wikis,  Biogs,  Apps,  etc.) 


Performance  Support  Apps 

(Mobiie  Digitai  Devices) 

Peer-Based  Learning 

(Digitai  Sociai  Networks) 


Context-based, 
facilitated,  problem 
solving  team  exercises 


Blended  Learning 


Regional  Learning  Centers 

(Sateliite  schoois  at  unit  iocations) 


Adaptive  Learning, 
Intelligent  Tutors 


Mobile  Learning, 
dL  Modules 


Assessments, 

Evaluations 


(Rigor  &  Relevance) 


Self-Structured  Tracking  &  Feedback 

Learning  (Army  career  Tracker) 


Enables  Career-Long  Learning  -  Supports  a  Learning  Army  i 


Figure  9.  Learner-Centric  2015  Learning  Environment  (From;  TRADOC,  2011) 
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Data  Quality 


Figure  10.  A  Hierarchy  of  Data  Quality  Dimensions  (After:  Wang  et  al.,  1992,  p.  3) 


43 


THIS  PAGE  INTENTIONALLY  LEET  BLANK 


44 


LIST  OF  REFERENCES 


Bearden,  R.  (2012).  Pine  Bluff  Arsenal.  In  The  Encyclopedia  of  Arkansas  history  & 
culture.  Retrieved  from  http  ://enevclopediaofarkansas  .net/ encyelopedia/ entry- 
detail. aspx?entrvlD=2927 

Blue  Grass  Army  Depot  (BGAD).  (n.d.).  Blue  Grass  Army  Depot  [Broehure].  Retrieved 
from  http://www.bluegrass.armv.mil/Info/BGAD%20Brochure.pdf 

Butler,  J.  (1998,  July).  A  history  of  information  technology  and  systems.  Retrieved  from 
http://www.tcfua.edu/AZ/ITHistorvOutline.htm 

Caterinicchia,  D.  (2002,  November).  Army  logistics  marches  ahead.  Retrieved  from 
http://www.fcw.eom/fcw/articles/2002/l  1 18/pol-armv-l  l-18-02.asp 

Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS).  (1997).  Joint  vision  2010.  Washington, 
DC:  Office  of  the  Chairman. 

Crane  Army  Ammunition  Activity,  (n.d.-a).  About  Crane  Army  Ammunition  Activity. 
Retrieved  from  http://www.crane.armv.mil/pa/historv.html 

Crane  Army  Ammunition  Activity,  (n.d.-b).  Letterkenny  Munitions  Center.  Retrieved 
from  http://www.crane.armv.mil/pa/lemc.html 

Department  of  the  Army.  (2007,  August  3).  Army  training  and  leader  development 
(Army  Regulation  350-1).  Washington,  DC:  Author. 

Department  of  the  Army.  (2010,  August  19).  The  United  States  Army  operating  concept: 
2016-2028  (TRADOC  Pamphlet  525-3-1).  Fort  Monroe,  VA:  Author. 

Government  Accountability  Office  (GAO).  (2010,  April).  Defense  logistics  actions 
needed  to  improve  implementation  of  the  Army  logistics  modernization  program 
(GAO- 10-461).  Retrieved  from  http://www.gao.gov/products/GAO-10-461 

Government  Accountability  Office  (GAO),  (n.d.).  About  GAO.  Retrieved  from 
http://www.gao.gov/about/index.html 

Hill,  C.  W.  (2007,  September).  Transforming  the  force:  A  comparative  analysis  of  the 
Department  of  Defense’s  (DOD’s)  enterprise  resource  planning  (ERP)  systems. 
Unpublished  master’s  thesis.  Naval  Postgraduate  School,  Monterey,  CA. 

Joint  Munitions  Command  (JMC).  (2012,  February  27).  Joint  Munitions  Command 
history.  Retrieved  from  http://www.imc.armv.mil/Historian/JMCHistorv.aspx 


45 


LMP  Special  Project  Team.  (2004).  The  business  case:  Wholesale  logistics 
modernization  program.  Fort  Belvoir,  VA:  U.S.  Army  CECOM. 

Means,  B.,  Toyama,  Y.,  Murphy,  R.,  Bakia,  M.,  Jones,  K.  (2010).  Evaluation  of 
evidenced-based  practices  in  online  learning:  A  meta-analysis  and  review  of 
online  learning  studies.  Washington,  DC:  Department  of  Education. 

McAlester  Army  Ammunition  Plant  (MCAAP).  (n.d.).  Products  and  services.  Retrieved 
from  http  ://www .  me  aap .  army .  mil/PN S/PN S .  aspx 

Office  of  the  Secretary  of  Defense  (OSD).  (1997a,  May).  Quadrennial  defense  review 
report.  Washington,  DC:  Department  of  Defense. 

Office  of  the  Secretary  of  Defense  (OSD).  (1997b).  Defense  reform  initiative. 
Washington,  DC:  Author. 

Plifka,  J.  (2011).  Blended  learning:  The  Army’s  future  in  education,  training,  and 
development  (Master’s  thesis,  U.S.  Army  War  College).  Retrieved  from 
http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA560138 

Tooele  Army  Depot,  (n.d.).  Tooele  Army  Depot’s  vision/mission  statement.  Retrieved 
from  http://www.tead.armv.mil/ 

United  States  Army  Training  and  Doctrine  Command  (TRADOC).  (n.d.).  About 
TRADOC.  Retrieved  from  http://www.tradoc.armv.mil/About.asp 

United  States  Army  Training  and  Doctrine  Command  (TRADOC).  (2011).  The  U.S. 
Army  learning  concept  for  2015  (TRADOC  Pam  525-8-2).  Retrieved  from 
http://www.tradoc.armv.mil/tpubs/pams/tp525-8-2.pdf 

Vandana,  S.  (2003,  April).  Eederal  IT  is  stable  despite  changes.  Retrieved  from 
http://www.gcn.com/print/22  8/21 733- l.html 

Wallace,  T.  E.  (2001).  ERP:  Making  it  happen:  The  implementer’s  guide  to  success  with 
enterprise  resource  planning.  New  York,  NY:  Wiley. 

Wang,  R.  Y.,  Reddy,  M.  P.,  &  Kon,  H.  B.  (1992).  Data  quality  requirements  analysis  and 
modeling.  Retrieved  from  http ://archive. org/ details/ dataqualitvrequi92wang 


46 


INITIAL  DISTRIBUTION  LIST 


1.  Defense  Teehnieal  Information  Center 
Ft.  Belvoir,  Virginia 

2.  Dudley  Knox  Library 
Naval  Postgraduate  Sehool 
Monterey,  California 


47 


